Appearance-based localization for mobile robots using digital zoom and visual compass

This paper describes a localization system for mobile robots moving in dynamic indoor environments, which uses probabilistic integration of visual appearance and odometry information. The approach is based on a novel image matching algorithm for appearance-based place recognition that integrates digital zooming, to extend the area of application, and a visual compass. Ambiguous information used for recognizing places is resolved with multiple hypothesis tracking and a selection procedure inspired by Markov localization. This enables the system to deal with perceptual aliasing or absence of reliable sensor data. It has been implemented on a robot operating in an office scenario and the robustness of the approach demonstrated experimentally

Efficient exploration of unknown indoor environments using a team of mobile robots

Whenever multiple robots have to solve a common task, they need to coordinate their actions to carry out the task efficiently and to avoid interferences between individual robots. This is especially the case when considering the problem of exploring an unknown environment with a team of mobile robots. To achieve efficient terrain coverage with the sensors of the robots, one first needs to identify unknown areas in the environment. Second, one has to assign target locations to the individual robots so that they gather new and relevant information about the environment with their sensors. This assignment should lead to a distribution of the robots over the environment in a way that they avoid redundant work and do not interfere with each other by, for example, blocking their paths. In this paper, we address the problem of efficiently coordinating a large team of mobile robots. To better distribute the robots over the environment and to avoid redundant work, we take into account the type of place a potential target is located in (e.g., a corridor or a room). This knowledge allows us to improve the distribution of robots over the environment compared to approaches lacking this capability. To autonomously determine the type of a place, we apply a classifier learned using the AdaBoost algorithm. The resulting classifier takes laser range data as input and is able to classify the current location with high accuracy. We additionally use a hidden Markov model to consider the spatial dependencies between nearby locations. Our approach to incorporate the information about the type of places in the assignment process has been implemented and tested in different environments. The experiments illustrate that our system effectively distributes the robots over the environment and allows them to accomplish their mission faster compared to approaches that ignore the place labels

Foley Music: Learning to Generate Music from Videos

In this paper, we introduce Foley Music, a system that can synthesize plausible music for a silent video clip about people playing musical instruments. We first identify two key intermediate representations for a successful video to music generator: body keypoints from videos and MIDI events from audio recordings. We then formulate music generation from videos as a motion-to-MIDI translation problem. We present a Graph$-$Transformer framework that can accurately predict MIDI event sequences in accordance with the body movements. The MIDI event can then be converted to realistic music using an off-the-shelf music synthesizer tool. We demonstrate the effectiveness of our models on videos containing a variety of music performances. Experimental results show that our model outperforms several existing systems in generating music that is pleasant to listen to. More importantly, the MIDI representations are fully interpretable and transparent, thus enabling us to perform music editing flexibly. We encourage the readers to watch the demo video with audio turned on to experience the results.Comment: ECCV 2020. Project page: http://foley-music.csail.mit.ed

Omnidirectional Vision for Appearance-based Robot Localization

Mobile robots need an internal representation of their environmA t to do useful things. Usually such a representation issom sort of geomjkkA m del. For our robot, which is equipped with a panoramA vision system we choose an appearancem del in which the sensoric data (in our case the panoramA imram have to bem odeled as a function of the robot position. Because imuse are veryhigh-dimjkzzxAm vectors, a feature extraction is needed before the m deling step. Very often a linear dimVkx:A reduction is used where the projection mtion is obtainedfrom a PrincipalCom onent Analysis (PCA). PCA isoptimx for the reconstruction of the data, but not necessarily the best linear projection for the localization task. We derived am ethod which extracts linear features optimI with respect to a riskm easure reflecting the localization performwjB) We tested themAjB d on a real navigation problem and comxBxfl it with an approach where PCAfeatures were used. 1 Intro ductio An inteqqD mo ofthe e nvironme t isneWMW to navigate amobile robot optimally from a curre tstate toward adePqDP state Such modeP can be topological maps, base on lab ebS rebSCPD tations for objeqx andtheW spatialrealSWxCq orge:Cqq:S modeC such as polygons or occupancy grids in the task space ofthe robot. Awide varie y of probabilisticmeob ds have be: de eO e to obtain a robuste stimate ofthe location ofthe robot give itsseSWqI inputs andthe e nvironme t mode: Theq meCC dsgeCMq:MS incorporate some obse ation mo de which give the probability of the sexIC meICMS0WI t give the location ofthe robot and the parame0WDC:M e vironme t modeM Some::S0 this parame:S vePDW deWIS0 e explicit propeopSW of the e vironme t (such as positions of landmarks [8] or occupancy value [4]) but can alsodeoSWC e an implicit reAKrK be twe: a seCWD ..